Unfooling Perturbation-Based Post Hoc Explainers

Monumental advancements in artificial intelligence (AI) have lured the interest of doctors, lenders, judges, and other professionals. While these high-stakes decision-makers are optimistic about technology, those familiar with AI systems wary lack transparency its decision-making processes. Perturbation-based post hoc explainers offer a model agnostic means interpreting while only requiring query-level access. However, recent work demonstrates that can be fooled adversarially. This discovery has adverse implications for auditors, regulators, sentinels. With this mind, several natural questions arise - how we audit black box systems? And ascertain auditee is complying good faith? In work, rigorously formalize problem devise defense against adversarial attacks on perturbation-based explainers. We propose algorithms detection (CAD-Detect) (CAD-Defend) attacks, which aided by our novel conditional anomaly approach, KNN-CAD. demonstrate approach successfully detects whether system adversarially conceals process mitigates attack real-world data prevalent explainers, LIME SHAP. The code available at https://github.com/craymichael/unfooling.